Skip to content

feat: add caselaw benchmark snapshot#1

Merged
blankey1337 merged 1 commit into
mainfrom
codex/evidencebench-caselaw
Jul 14, 2026
Merged

feat: add caselaw benchmark snapshot#1
blankey1337 merged 1 commit into
mainfrom
codex/evidencebench-caselaw

Conversation

@blankey1337

Copy link
Copy Markdown
Member

Summary

  • adds a sealed 16-item caselaw module balanced across federal/state, civil/criminal, and popular/obscure decisions
  • adds reporter-citation validation and dimension-aware scoring
  • publishes aggregate-only v2.0.0 results for six pinned OpenRouter models

Validation

  • 112 holdout records and 112 unique IDs for every model
  • 9 evaluator tests
  • dataset/schema validation
  • private holdout and raw outputs remain gitignored

@blankey1337
blankey1337 merged commit 426d093 into main Jul 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants